The parent application Ser. No. 09/310,715 included a three part microfiche appendix which is a listing of code for computer programs: There are a total of 36 microfiche with a total of 2451 frames. The microfiche appendix filed in application Ser. No. 09/310,715 is hereby incorporated herein by reference.
The present invention relates to photography, digital image processing and to computer graphics. More particularly the present invention relates to a method and system for providing a viewer with a multidimensional view which simulates movement through space or time.
A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
Motion is usually simulated by means of single view movies. Single view movies consist of a series of single photographs which are sequentially projected on a screen. At any time, a single photograph is entirely projected on the screen. Some movie theaters have screens which partially surround the viewer, so that a viewer can turn and look at a different portion of the screen as the movie progresses However, irrespective of where the viewer""s attention is focused, in fact the entire image is being projected on the screen. With some equipment single view movies can be stopped and reversed; however, again at a particular time a selected frame is entirely displayed. In summary, traditional movies do not allow a viewer to control the portion of the image which is projected on the screen. Stated differently, with traditional movies a viewer can not control the xe2x80x9cview windowxe2x80x9d through which each image is viewed.
It should be noted that as used herein the term xe2x80x9cscreenxe2x80x9d refers to either a traditional screen onto which an image is projected or an electronic display which projects or displays an image in such a manner that the image can be seen by a viewer.
The technology for producing panoramic images and photographs is well known. Panoramic images are images which represent the visual surroundings from a single location (or view point) of a particular 3D environment. Panoramic images can be photographic, computer-generated (CG), or a composite of photo and CG imagery. Equipment is available which seams together a series of two dimensional conventional images to form a panoramic image. Panoramic images may consist of any field of view such as a full sphere full cylinder, semi-sphere, etc. however, full sphere views are generally preferred. Panoramic images may be in any of the known projection formats, such as equi-rectangular, Mercator, Peters, fisheye, cube, or hemicube, etc. If the field of view is wide enough to warrant, perspective correction may be applied to the portion of a panoramic image displayed in order to remove projection distortions from the given view of the user. Computer programs and systems which allow one to view a selected portion of, to pan, to rotate, etc a panoramic image or photograph in response to the movement of a computer mouse, joystick, keyboard, etc. are commercially available.
Panoramic image (or photographic image) viewing systems are available which provide a number of panoramas (for example views of adjacent rooms in a building) and which allow a user who is viewing one of the rooms to xe2x80x9cclickxe2x80x9d on a door and to thus bring up the panorama of the next room, thereby, in some sense simulating movement into the next room. However, each of the panoramic views in such systems is static and some explicit action on the part of the viewer is required to move on to a different panorama.
U.S. Pat. Nos. 5,023,925 and 5,703,604 describe, and Dodeca L.L.C. located in Portland Oreg. commercially markets, a system for capturing images using a multi lens camera, for seaming the images into panoramas, and for viewing selected portions of the panoramic images.
Other panoramic image or photographic viewing systems are available which initiate a conventional single view motion picture when a user xe2x80x9cclicksxe2x80x9d on a spot such as a door in a first panoramic view. The single view movie simulates movement into a different location and at the end of the movie, the user is presented with a second panoramic view.
A company named xe2x80x9cWarpxe2x80x9d located in the Kingdom of Tonga has demonstrated a system wherein a video sequence is captured using a video camera with a fisheye lens which is pointed in the vertical or xe2x80x9cupxe2x80x9d direction (see, VTV Software Development Kit Reference Manual 2.01 Win95 1996). This approach provides a hemispheric movie in which the user may xe2x80x9cpan aroundxe2x80x9d while the movie is playing. In the system demonstrated by Warp, the user views the movie xe2x80x9cin sequencexe2x80x9d, meaning that each frame of the movie is played according to the temporal sequence in which the hemispheric movie was captured. The system demonstrated by Warp was limited to sub-spherical panorama dimensions and the camera was located at a fixed position.
Realistic simulation of movement from one location to another can be provided by three dimensional computer modeling systems, such as those used in some flight simulators. However, such systems are very computationally intensive
The present invention simulates movement through multi-dimensional space using a series of panoramic images which are projected or displayed in sequence. The user""s direction of view, that is the selected view window, is maintained as the series of images is projected or displayed. Motion in directions other than forward or reverse is simulated by utilizing xe2x80x9cbranchxe2x80x9d points in the sequence. Each path from a branch point can simulate motion in a different direction. Branch points are generally indicated to a viewer by visual indicators called xe2x80x9chot spotsxe2x80x9d; however, branch points may also be hidden and activated in response to the viewer""s selected direction of view. If a branch point is indicated by a visual indicator, a user can select motion in a desired direction by xe2x80x9cclickingxe2x80x9d on a xe2x80x9chot spotxe2x80x9d.
In order to conserve storage space, the image representing each panorama can be stored in a compressed format. If the images are stored in compressed format, in order to conserve time and processing power, when an image is displayed, only the portion of the panorama necessary to create a xe2x80x9cview windowxe2x80x9d that is, the portion of the image displayed in response to the user""s direction of view, is decompressed. Furthermore, the images are stored in a format that does not utilize inter-image compression (such as that used by the MPEG format). Since the images are stored in a format that does not utilize inter-image compression, it is possible to simulate motion in both the forward and backward direction without operating on a series of decompressed images.
An index methodology is used to store the panoramic images. Use of the indexing methodology allows the images to be retrieved in both the forward and backward direction to simulate movement in either direction.
Sound is provided in a special format, so that special effects can be provided based on the user""s point of view and dependent upon the direction of motion selected by the user.